Method and apparatus for encoding and decoding stereoscopic video

ABSTRACT

Stereoscopic video is encoded and decoded by using the MAC defined by the existing MPEG-4 standard. The stereoscopic video is divided into one image as a single video object, and another image as auxiliary information for the image established as the video object. The auxiliary information includes a horizontal disparity map, a vertical disparity map, luminance residual texture, and chrominance residual texture, which are respectively allocated to auxiliary components of the MAC according to importance and complexity of the images, encoded, and then output as a single encoding stream.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of Korea PatentApplication No. 2003-84724 filed on Dec. 27, 2002 in the KoreanIntellectual Property Office, the content of which is incorporatedherein by reference.

BACKGROUND OF THE INVENTION

(a) Field of the Invention

The present invention relates to a method and device for encoding anddecoding stereoscopic video. More specifically, the present inventionrelates to a method and device for encoding and decoding stereoscopicvideo into an encoding stream by using a conventional MPEG-4 MAC(multiple auxiliary component.)

(b) Description of the Related Art

A MVP (multi-view profile) method for extending two-dimensional videoencoding techniques in the MPEG-2 technology has been developed as aconventional stereoscopic video encoding method. As to the MVP method,an encoding structure of a base layer for performing encoding by usingmotion compensation corresponds to that of the MPEG-2 MP (main profile),and hence, one of the right and left images is reconstructed when theconventional 2-dimensional video decoder reconstructs the data of thebase layer, thereby maintaining compatibility with the existing2-dimensional video decoder system. An encoder of an enhancement layeruses correlation information provided between the right and left imagesto perform encoding. This is a method for performing encoding by usingtemporal scalability, and the MPEG-4 standard has also defined thetemporal scalability using stereoscopic video encoding.

The above-noted prior art has been disclosed by U.S. Pat. No. 5,612,735entitled “Digital 3D/stereoscopic video compensation technique utilizingtwo disparity estimates.” Regarding the '735 patent on the temporalscalability basis, a base layer uses a motion compensation algorithm andDCT (discrete cosine transform) base algorithm to encode images of theleft eye, and an enhancement layer uses disparity information of betweenthe base layer and the enhancement layer to encode the images of theright eye without motion compensation for the images of the right eye.

Also, as to the U.S. Pat. No. 5,619,256 entitled “Digital3D/stereoscopic video compensation technique utilizing disparity andmotion compensated predictions,” the base layer uses a motioncompensation algorithm and DCT base algorithm to encode images of theleft eye, and the enhancement layer uses information on the motioncompensation between the images of the right eye and disparity ofbetween the base layer and the enhancement layer to encode the images ofthe right eye on the temporal scalability basis in the like manner ofthe '735 patent. This method achieves efficient compression rates byperforming encoding by using motion and disparity information, but themethod has a complicated encoding structure and it is difficult torealize the method in the hardwired manner, and the method requires alarge amount of calculation when processing HDTV images.

Therefore, in the case of encoding stereoscopic video by using temporalscalability, an additional multiplexer for transmitting respectiveencoding streams output by the base layer and the enhancement layer as asingle stream is problematically needed in order to simplify asynchronization problem between the right and left images.

Conventional multi-view video encoding methods include a method forperforming encoding by using a disparity map having disparity vectorvalues of pixels, which has been disclosed by U.S. Pat. No. 6,055,274entitled “Method and apparatus for compressing multi-view video.”

The '274 patent encodes the total image data of the first image (a leftimage), and generates a disparity map having disparity vector values forrespective pixels from the first image and a second image (a rightimage) to perform encoding on the motion compensated disparity vectors,and uses a reconstructed first image after the encoding todisparity-compensate for a reconstructed disparity map and encoderesidual image data of between the generated second image and the inputsecond original image. This method problematically outputs a pluralityof encoding streams, and additionally requires a multiplexer fortransmitting them in a single stream format.

In order to use the conventional MPEG codecs for 2-dimensional images,and perform simple synchronization between the right and left images inthe stereoscopic video, a method for reducing the respective right andleft images by ½ and converting them into 2-dimensional standard imageshas been proposed in five methods in the transactions of “3D videostandards conversion, stereoscopic displays and applications,” by AndrewWoods, Tom Docherty, and Rolf Koch (VII, California, February 1996,Proceedings of the SPIE vol. 1653A).

The above technique is also published in U.S. Pat. No. 5,633,682entitled “Stereoscopic coding system.”

The invention published as '682 selects odd-field images for the imagesof the left eye and even-field images for the images of the right eye toconvert them into a single image, and accordingly performs MPEG encodingon the converted single image with respect to the existing 2-dimensionalimages. This method considers a shuttering method for alternatelydisplaying the right and left images in the case of displaying thestereoscopic video, and the method is not suitable for polarizeddisplays that concurrently display the right and left images.

SUMMARY OF THE INVENTION

It is an advantage of the present invention to maintain compatibilityfor using existing MPEG-4 encoding techniques and systems, and minimizecomplexity of synchronization between the right and left images.

It is another advantage of the present invention to selectively controlquality of images and encode the images according to importance orcomplexity of the images, thereby improving encoding efficiencies.

In the first aspect of the present invention, a method for encodingstereoscopic video including first and second images comprises: (a)encoding the first image, and outputting a quantized video object and amotion vector of the first image; (b) receiving the first and secondimages, and finding a pixel-based horizontal disparity map on the secondimage with reference to the first image; and (c) encoding the horizontaldisparity map and outputting a quantized horizontal disparity map basedon the pixel-based horizontal disparity map and a motion vector.

In the second aspect of the present invention, a method for encodingstereoscopic video including first and second images comprises: (a)encoding the first image, and outputting a quantized video object and amotion vector of the first image; (b) decoding the quantized videoobject output in (a), and reconstructing the first image; (c) receivingthe first and second images, and finding a pixel-based horizontaldisparity map on the second image with reference to the first image; (d)encoding the horizontal disparity map and outputting a quantizedhorizontal disparity map based on the pixel-based horizontal disparitymap and the motion vector; (e) reconstructing the quantized horizontaldisparity map output in (d), and outputting a reconstructed horizontaldisparity map; (f) performing disparity compensation and outputting apixel value of a disparity-compensated second image based on a pixelvalue of the first image reconstructed in (b) and a horizontal disparityvector value of the horizontal disparity map reconstructed in (e); and(g) performing a residual process on the pixel value of the second imageand the pixel value of the disparity-compensated second image output in(f) to output luminance residual texture, and encoding the luminanceresidual texture to output quantized luminance residual texture.

In the third aspect of the present invention, a method for encodingstereoscopic video including first and second images comprises: (a)encoding the first image, and outputting a quantized video object and amotion vector of the first image; (b) decoding the quantized videoobject output in (a), and reconstructing the first image; (c) receivingthe first and second images, and finding a pixel-based horizontaldisparity map and a pixel-based vertical disparity map on the secondimage with reference to the first image; (d) encoding the horizontaldisparity map and outputting a quantized horizontal disparity map basedon the pixel-based horizontal disparity map and the motion vector; (e)encoding the vertical disparity map and outputting a quantized verticaldisparity map based on the pixel-based vertical disparity map and themotion vector; (f) reconstructing the quantized horizontal disparity mapoutput in (d), and outputting a reconstructed horizontal disparity map;(g) reconstructing the quantized vertical disparity map output in (d),and outputting a reconstructed vertical disparity map; (h) performingdisparity compensation and outputting a pixel value of adisparity-compensated second image based on a pixel value of the firstimage reconstructed in (b), a horizontal disparity vector value of thehorizontal disparity map reconstructed in (f), and a vertical disparityvector value of the vertical disparity map reconstructed in (h); and (i)performing a residual process on the pixel value of the second image andthe pixel value of the disparity-compensated second image output in (h)to output luminance residual texture, and encoding the luminanceresidual texture to output quantized luminance residual texture.

In the fourth aspect of the present invention, a method for decodingstereoscopic video including first and second images comprises: (a)receiving an encoding stream, and outputting quantized data of a videoobject of the first image, a motion vector, and quantized data of ahorizontal disparity map; (b) decoding the video object andreconstructing the first image based on the quantized data of the videoobject and the motion vector; (c) decoding the quantized data of thehorizontal disparity map based on the quantized data of the horizontaldisparity map and the motion vector; and (d) performing disparitycompensation based on the reconstructed first image and the decodedhorizontal disparity map, and reconstructing the second image.

In the fifth aspect of the present invention, a method for decodingstereoscopic video including first and second images comprises: (a)receiving an encoding stream, and outputting quantized data of a videoobject of the first image, a motion vector, quantized data of ahorizontal disparity map, and quantized data of luminance residualtexture; (b) decoding the video object and reconstructing the firstimage based on the quantized data of the video object and the motionvector; (c) decoding the quantized data of the horizontal disparity mapbased on the quantized data of the horizontal disparity map and themotion vector; (d) decoding the quantized data of the luminance residualtexture based on the quantized data of the luminance residual textureand the motion vector; (e) performing disparity compensation based onthe reconstructed first image and the decoded horizontal disparity map,and outputting disparity-compensated luminance texture; and (f) addingthe disparity-compensated luminance texture and the luminance residualtexture reconstructed in (d) to reconstruct the second image.

In the sixth aspect of the present invention, a method for decodingstereoscopic video including first and second images comprises: (a)receiving an encoding stream, and outputting quantized data of a videoobject of the first image, a motion vector, quantized data of ahorizontal disparity map, quantized data of a vertical disparity map,and quantized data of luminance residual texture; (b) decoding the videoobject and reconstructing the first image based on the quantized data ofthe video object and the motion vector; (c) decoding the quantized dataof the horizontal disparity map based on the quantized data of thehorizontal disparity map and the motion vector; (d) decoding thequantized data of the vertical disparity map based on the quantized dataof the vertical disparity map and the motion vector; (e) decoding thequantized data of the luminance residual texture based on the quantizeddata of the luminance residual texture and the motion vector; (f)performing disparity compensation based on the reconstructed firstimage, the decoded horizontal disparity map, and the decoded verticaldisparity map, and outputting disparity-compensated luminance texture;and (g) adding the disparity-compensated luminance texture and theluminance residual texture reconstructed in (e) to reconstruct thesecond image.

In the seventh aspect of the present invention, an encoder forstereoscopic video including first and second images comprises: a videoobject encoder for encoding the first image, and outputting a quantizedvideo object and a motion vector of the first image; a disparityestimator for receiving the first and second images, and finding apixel-based horizontal disparity map on the second image with referenceto the first image; and an auxiliary component encoder for encoding thehorizontal disparity map and outputting a quantized horizontal disparitymap based on the pixel-based horizontal disparity map output by thedisparity estimator and a motion vector output by the video objectencoder.

In the eighth aspect of the present invention, an encoder forstereoscopic video including first and second images comprises: a videoobject encoder for encoding the first image to output a quantized videoobject and a motion vector of the first image, and encoding thequantized video object to output a reconstructed first image; adisparity estimator for receiving the first and second images, andfinding a pixel-based horizontal disparity map on the second image withreference to the first image; a first auxiliary component encoder forencoding the horizontal disparity map to output a quantized horizontaldisparity map, and decoding the output and quantized horizontaldisparity map to output a reconstructed horizontal disparity map basedon the pixel-based horizontal disparity map output by the disparityestimator and the motion vector output by the video object encoder; adisparity compensator for performing disparity compensation andoutputting a pixel value of a disparity-compensated second image basedon a pixel value of the reconstructed first image output by the videoobject encoder and a horizontal disparity vector value of thereconstructed horizontal disparity map output by the first auxiliarycomponent encoder; and a second auxiliary component encoder forperforming a residual process on the pixel value of the second image andthe pixel value of the disparity-compensated second image output by thedisparity compensator to output luminance residual texture, and encodingthe luminance residual texture to output quantized luminance residualtexture.

In the ninth aspect of the present invention, a decoder for stereoscopicvideo including first and second images comprises: a variable lengthdecoder for receiving an encoding stream, and outputting quantized dataof a video object of the first image, a motion vector, and quantizeddata of a horizontal disparity map; a video object decoder for decodingthe video object and reconstructing the first image based on thequantized data of the video object and the motion vector output by thevariable length decoder; an auxiliary component decoder for decoding thequantized data of the horizontal disparity map based on the quantizeddata of the horizontal disparity map and the motion vector output by thevariable length decoder; and disparity compensator for performingdisparity compensation based on the reconstructed first image output bythe video object decoder and the decoded horizontal disparity map outputby the auxiliary component decoder, and reconstructing the second image.

In the tenth aspect of the present invention, a decoder for stereoscopicvideo including first and second images comprises: a variable lengthdecoder for receiving an encoding stream, and outputting quantized dataof a video object of the first image, a motion vector, quantized data ofa horizontal disparity map, and quantized data of luminance residualtexture; a video object decoder for decoding the video object andreconstructing the first image based on the quantized data of the videoobject and the motion vector; a first auxiliary component decoder fordecoding the quantized data of the horizontal disparity map based on thequantized data of the horizontal disparity map and the motion vector; asecond auxiliary component decoder for decoding the quantized data ofthe luminance residual texture based on the quantized data of theluminance residual texture and the motion vector; a disparitycompensator for performing disparity compensation based on thereconstructed first image output by the video object decoder and thedecoded horizontal disparity map output by the first auxiliary componentdecoder, and outputting disparity-compensated luminance texture anddisparity-compensated chrominance texture; and a first adder for addingthe disparity-compensated luminance texture output by the disparitycompensator and the reconstructed luminance residual texture output bythe second auxiliary component decoder.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate an embodiment of the invention,and, together with the description, serve to explain the principles ofthe invention:

FIG. 1 shows types and numbers of auxiliary components ofvideo_object_layer_shape_extension defined by the MPEG-4 standards;

FIG. 2 shows a stereoscopic video encoder according to a first preferredembodiment of the present invention;

FIG. 3 shows a stereoscopic video decoder according to a first preferredembodiment of the present invention;

FIG. 4 shows types and numbers of auxiliary components ofvideo_object_layer_shape_extension which requires additional definitionfor encoding the stereoscopic video according to a preferred embodimentof the present invention;

FIG. 5 shows types and numbers of auxiliary components ofvideo_object_layer_shape_extension which requires additional definitionwhen increasing the number of auxiliary components of the MAC (multipleauxiliary component) to greater than four;

FIG. 6 shows a stereoscopic video encoder according to a secondpreferred embodiment of the present invention;

FIG. 7 shows a stereoscopic video decoder according to a secondpreferred embodiment of the present invention;

FIG. 8 shows an encoding stream output by the encoder according to asecond preferred embodiment of the present invention;

FIG. 9 shows a stereoscopic video encoder according to a third preferredembodiment of the present invention;

FIG. 10 shows a stereoscopic video decoder according to a thirdpreferred embodiment of the present invention;

FIG. 11 shows an encoding stream output by the encoder according to athird preferred embodiment of the present invention;

FIG. 12 shows a stereoscopic video encoder according to a fourthpreferred embodiment of the present invention;

FIG. 13 shows a stereoscopic video decoder according to a fourthpreferred embodiment of the present invention;

FIG. 14 shows an encoding stream output by the encoder according to afourth preferred embodiment of the present invention;

FIG. 15 shows a stereoscopic video encoder according to a fifthpreferred embodiment of the present invention;

FIG. 16 shows a stereoscopic video decoder according to a fifthpreferred embodiment of the present invention;

FIG. 17 shows an encoding stream output by the encoder according to afifth preferred embodiment of the present invention;

FIG. 18 shows a detailed block diagram for a video object encoderaccording to a preferred embodiment of the present invention;

FIG. 19 shows a detailed block diagram for an auxiliary componentencoder according to a preferred embodiment of the present invention;

FIG. 20 shows a detailed block diagram for a video object decoderaccording to a preferred embodiment of the present invention; and

FIG. 21 shows a detailed block diagram for an auxiliary componentdecoder according to a preferred embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following detailed description, only the preferred embodiment ofthe invention has been shown and described, simply by way ofillustration of the best mode contemplated by the inventor(s) ofcarrying out the invention. As will be realized, the invention iscapable of modification in various obvious respects, all withoutdeparting from the invention. Accordingly, the drawings and descriptionare to be regarded as illustrative in nature, and not restrictive.

The current MPEG-4 codec allocates auxiliary information includingalpha, disparity, and depth which relate to respective video objects toan MAC (multiple auxiliary component) including three auxiliarycomponents to encode them.

The present invention relates to a method for using the MPEG-4 MAC toencode the stereoscopic video, and auxiliary information to be allocatedto the MAC will be defined.

The current MPEG-4 MAC is added to MPEG-4 visual version 2 so as todescribe transparency of the video objects, and further, it definesauxiliary information including disparity, depth, and additional texturerelating to the video objects.

FIG. 1 shows types and numbers of auxiliary components ofvideo_object_layer_shape_extension defined by the MPEG-4 standards.

As shown, values of video_object_layer_shape_extension in which theauxiliary components include the disparity type are defined as 0001,0010, 0011, and 0100, and no auxiliary components are defined to thevalues of from 1101 to 1111.

In the preferred embodiment, undefined auxiliary components ofvideo_object_layer_shape_extension values are defined to include neededauxiliary information for more effective encoding of stereoscopic video.

FIG. 2 shows a stereoscopic video encoder according to a first preferredembodiment of the present invention. The encoder uses auxiliarycomponents defined by the existing MPEG-4 MAC shown in FIG. 1 to encodethe stereoscopic video data.

As shown, the encoder comprises a disparity estimator 100, a videoobject encoder 200, an auxiliary component encoder 320, and a variablelength encoder 600.

The video object encoder 200 receives one image (a left image in thepreferred embodiment) of the stereoscopic video, and outputs a quantizedvideo object and a motion vector. The disparity estimator 100 receivesleft and right images, and finds a pixel-based horizontal disparity mapof the right image with reference to the left image. That is, thedisparity estimator 100 outputs position vectors moved to the horizontalaxis as a horizontal disparity map so as to search and estimate to whatpositions of the left image the pixels of the right image are provided.

The auxiliary component encoder 320 receives the horizontal disparitymap which is an auxiliary component output by the disparity estimator100 and the motion vector output by the video object encoder 200, andoutputs a quantized horizontal disparity map.

The variable length encoder 600 receives the quantized video object andthe motion vector output by the video object encoder 200 and thequantized horizontal disparity map output by the auxiliary componentencoder 320, performs variable length encoding, and outputs an encodingstream.

FIG. 3 shows a decoder according to a first preferred embodiment of thepresent invention.

As shown, the decoder comprises a variable length decoder 700, a videoobject decoder 800, an auxiliary component decoder 920, and a disparitycompensator 1000.

The variable length decoder 700 performs variable length decoding on theencoding stream output by the encoder of FIG. 2, and outputs quantizeddata of the video object, a motion vector, and quantized data of thehorizontal disparity map.

The video object decoder 800 receives the quantized data of the videoobject and the motion vector, and decodes the video object to decode theleft image.

The auxiliary component decoder 920 receives the quantized data of thehorizontal disparity map and the motion vector, and decodes thehorizontal disparity map.

The disparity compensator 1000 receives a reconstructed left image and areconstructed horizontal disparity map, performs disparity compensationbased on a disparity vector of the horizontal disparity map, and finallyreconstructs a right image.

As described above, the disparity estimation is a process for searchingand estimating to what positions of the left image the pixels of theright image are provided, and position vectors moved to the horizontalaxis are output as a horizontal disparity map, while position vectorsmoved to the vertical axis are output as a vertical disparity map. Ingeneral, a disparity map represents the horizontal disparity map becausethe right and left images of the ideal stereoscopic image only havehorizontal disparity values, and all the vertical disparity values arezero.

The encoder and the decoder according to the first preferred embodimentare used for an ideal system which only has horizontal disparity valueswith the vertical disparity values of 0, and use the predefined MAC.

That is, the encoder and the decoder according to the preferredembodiment use the value of video_object_layer_shape_extension with theconventional auxiliary components including the disparity type of FIG. 1to allocate the horizontal disparity map to the disparity type auxiliarycomponent and encode the same. In this instance, the disparity map isfound for the luminance data from among the luminance data and thechrominance data.

Since the stereoscopic video by computer graphics are ideally generatedthrough a computer, the vertical disparity values can all be set to bezero so as to generate the stereoscopic video. Therefore, the accuratedisparity map can be found through the encoder and the decoder accordingto the first preferred embodiment, and the quality of the right image isguaranteed to some degrees by this information.

However, in the case of real images captured by a stereoscopic camera,the vertical disparity values are provided according to shapes andperformance of the actually manufactured stereoscopic camera differingfrom the images by the computer graphics. In addition, since theaccuracy of the disparity map by the horizontal disparity vectors foundby calculation is lowered below the accuracy of computer graphicsimages, the quality of the right image reconstructed only throughhorizontal disparity information is substantially worsened. Further,since the disparity map according to the first preferred embodiment hasno information on an occlusion area which is not provided to the leftimage but is to the right image, it is difficult to preciselyreconstruct the right image.

Accordingly, information on the vertical disparity and residual textureas well as information on the horizontal disparity is added asinformation on the right image to the MPEG-4 MAC to definevideo_object_layer_shape_extension having the auxiliary components ofFIGS. 4 and 5.

The video_object_layer_shape_extension of FIG. 4 includes values of from1101 to 1111 which are not defined by the existing MPEG-4 MAC but arenewly defined as an auxiliary component in the preferred embodiment ofthe present invention, and the video_object_layer_shape_extension ofFIG. 5 is newly defined in the preferred embodiment of the presentinvention so as to support four auxiliary components which are notsupported by the MPEG-4 MAC.

As shown in FIGS. 4 and 5, disparity information is divided into ahorizontal disparity map and a vertical disparity map by the horizontaland vertical disparity values, the residual texture is classified asluminance residual texture and chrominance residual texture information,and auxiliary information is selected according to importance andcomplexity of images to perform encoding. In this instance, theluminance residual texture information represents data of the left imagereconstructed after decoding, the right image disparity-compensated bythe reconstructed disparity map, and the residual image by the luminancecomponent of the input right image. The chrominance residual textureinformation represent data of the left image reconstructed afterencoding, the right image disparity-compensated by the reconstructeddisparity map, and the residual image by the chrominance component ofthe input right image.

FIGS. 6 and 7 respectively show an encoder and a decoder according to asecond preferred embodiment of the present invention. The encoder andthe decoder establish the video_object_layer_shape_extension which hastwo auxiliary components of the horizontal disparity type and theluminance residual texture type which are newly defined for encoding thestereoscopic video, and encode and decode the stereoscopic video.

As shown in FIG. 6, the encoder comprises a disparity estimator 100, avideo object encoder 200, auxiliary component encoders 320 and 340, adisparity compensator 400, an adder 500, and a variable length encoder600.

The video object encoder 200 receives one image (a left image in thepreferred embodiment) of the stereoscopic video, outputs a quantizedvideo object and a motion vector, and outputs a left image obtained byreconstructing the quantized video object.

The disparity estimator 100 receives left and right images, and finds apixel-based horizontal disparity map of the right image with referenceto the left image.

The auxiliary component encoder 320 receives the horizontal disparitymap which is an auxiliary component output by the disparity estimator100 and the motion vector output by the video object encoder 200,generates a quantized horizontal disparity map, outputs the quantizedhorizontal disparity map, and outputs a horizontal disparity mapobtained by reconstructing the quantized horizontal disparity map.

The disparity compensator 400 performs disparity compensation based onpixel values (luminance) of the reconstructed left image output by thevideo object encoder 200 and the horizontal disparity vectors of thereconstructed horizontal map output by the auxiliary component encoder320, and outputs pixel values (luminance) of the compensated rightimage.

The adder 500 performs a residual process on the pixel value (luminance)of the right image and the pixel value (luminance) of thedisparity-compensated right image output by the disparity compensator400 to output luminance residual texture, and the auxiliary componentencoder 340 encodes the luminance residual texture to output quantizedluminance residual texture.

The variable length encoder 600 performs variable length encoding on thequantized video object and the motion vector output by the video objectencoder 200, the quantized horizontal disparity map output by theauxiliary component encoder 320, and the quantized luminance residualtexture output by the auxiliary component encoder 340, and outputs anencoding stream.

FIG. 8 shows an encoding stream output by the encoder according to thesecond preferred embodiment of the present invention. As shown, theencoding stream output by the encoder (a variable length encoder)includes macro blocks of the encoded video object (the left image),macro blocks of an auxiliary component AC[0] for the encoded horizontaldisparity map, and macro blocks of an auxiliary component AC[1] forencoded luminance residual texture.

FIG. 7 shows a decoder according to a second preferred embodiment of thepresent invention.

As shown, the decoder comprises a variable length decoder 700, a videoobject decoder 800, auxiliary component decoders 920 and 940, adisparity compensator 1000, and an adder 1100.

The variable length decoder 700 performs variable length decoding on theencoding stream output by the encoder as shown in FIG. 8, and outputsquantized data of the video object, a motion vector, quantized data ofthe horizontal disparity map, and quantized data of the luminanceresidual texture.

The video object decoder 800 receives the quantized data of the videoobject and the motion vector, and decodes the video object to decode theleft image.

The auxiliary component decoder 920 receives the quantized data of thehorizontal disparity map and the motion vector, and decodes thehorizontal disparity map.

The auxiliary component decoder 940 receives the quantized data of theluminance residual texture and the motion vector, and decodes theluminance residual texture.

The disparity compensator 1000 receives a reconstructed left image and areconstructed horizontal disparity map, and performs disparitycompensation based on a disparity vector of the horizontal disparitymap.

The adder 1100 adds the disparity-compensated luminance texture fromamong the disparity-compensated data and the reconstructed luminanceresidual texture output by the auxiliary component decoder 940 toreconstruct the right image.

In the second preferred embodiment shown in FIGS. 6 and 7, the leftimage is encoded and decoded as a single video object, and the rightimage is encoded and decoded using the MAC shown in FIG. 4. That is, thepixel-based horizontal disparity map found with reference to the leftimage is allocated to aux_comp_type[0], and the luminance residualtexture which is residual image data of the luminance component isallocated to aux_comp_type[1] to perform encoding and decoding.

The second preferred embodiment is applicable to simple images which donot greatly influence deterioration of image quality when performingencoding without a vertical disparity map and chrominance residualtexture information which is residual image data on the chrominancecomponent, or to images which require no precise reconstruction.

FIGS. 9 and 10 respectively show an encoder and a decoder according to athird preferred embodiment of the present invention. The encoder and thedecoder establishes the video_object_layer_shape_extension which hasthree auxiliary components of the horizontal disparity type, theluminance residual texture type, and the chrominance residual texturetype which are additionally defined with reference to FIG. 4, andperforms encoding and decoding.

That is, the pixel-based horizontal disparity map found with referenceto the left image is allocated to aux_comp_type[0], the luminanceresidual texture is allocated to aux_comp_type[1], and the chrominanceresidual texture which is residual image data on the chrominancecomponent is allocated to aux_comp_type[2]. The encoder and the decoderare usable for simple images which do not greatly influencedeterioration of image quality when performing encoding without avertical disparity map, or for images which require no precisereconstruction.

As shown in FIG. 9, the encoder according to the third preferredembodiment of the present invention comprises a disparity estimator 100,a video object encoder 200, auxiliary component encoders 320, 340, and360, a disparity compensator 400, an adder 500, and a variable lengthencoder 600.

The components of FIG. 9 which perform the same or similar functions asthose of the encoder of FIG. 6 have the same reference numerals, and norepeated descriptions on the components of FIG. 9 which execute the sameoperations as those of the components of FIG. 6 will be provided.

The disparity compensator 400 performs disparity compensation based onpixel values (luminance and chrominance) of the reconstructed left imageoutput by the video object encoder 200 and the horizontal disparityvectors of the reconstructed horizontal map output by the auxiliarycomponent encoder 320, and outputs pixel values (luminance andchrominance) of the compensated right image.

The adder 500 performs a residual process on the pixel values (luminanceand chrominance) of the disparity-compensated right image and the pixelvalues (luminance and chrominance) of the compensated right image outputby the disparity compensator 400 to output luminance residual textureand chrominance residual texture, and the auxiliary component encoders340 and 360 encode the luminance residual texture and the chrominanceresidual texture to output quantized luminance residual texture andquantized chrominance residual texture.

The variable length encoder 600 performs variable length encoding on thequantized video object and the motion vector output by the video objectencoder 200, the quantized horizontal disparity map output by theauxiliary component encoder 320, the quantized luminance residualtexture output by the auxiliary component encoder 340, and the quantizedchrominance residual texture output by the auxiliary component encoder360, and outputs an encoding stream.

FIG. 11 shows an encoding stream output by the encoder according to thethird preferred embodiment of the present invention. As shown, theencoding stream output by the encoder includes macro blocks of theencoded video object, macro blocks of the auxiliary component AC[0] forthe encoded horizontal disparity map, macro blocks of the auxiliarycomponent AC[1] for the encoded luminance residual texture, and macroblocks of the auxiliary component AC[2] for the encoded chrominanceresidual texture.

FIG. 10 shows a decoder according to the third preferred embodiment ofthe present invention.

As shown, the decoder comprises a variable length decoder 700, a videoobject decoder 800, auxiliary component decoders 920, 940, and 960, adisparity compensator 1000, and adders 1100 and 1200.

The components of FIG. 10 which perform the same or similar functions asthose of the decoder of FIG. 7 have the same reference numerals, and norepeated descriptions on the components of FIG. 10 which execute thesame operations as those of the components of FIG. 7 will be provided.

The variable length decoder 700 performs variable length decoding on theencoding stream output by the encoder and shown in FIG. 11, and outputsquantized data of the video object, a motion vector, quantized data ofthe horizontal disparity map, quantized data of the luminance residualtexture, and quantized data of the chrominance residual texture.

The auxiliary component decoder 940 receives the quantized data of theluminance residual texture and the motion vector, and decodes theluminance residual texture.

The auxiliary component decoder 960 receives the quantized data of thechrominance residual texture and the motion vector, and decodes thechrominance residual texture.

The disparity compensator 1000 receives a reconstructed left image and areconstructed horizontal disparity map, and performs disparitycompensation based on a disparity vector of the horizontal disparitymap.

The adder 1100 adds the disparity-compensated luminance texture fromamong the data disparity-compensated by the disparity compensator 1000and the reconstructed luminance residual texture output by the auxiliarycomponent decoder 940 to reconstruct the luminance component of theright image.

The adder 1200 adds the disparity-compensated chrominance texture fromamong the disparity-compensated data and the reconstructed chrominanceresidual texture output by the auxiliary component decoder 940 toreconstruct the chrominance component of the right image.

FIGS. 12 and 13 respectively show an encoder and a decoder according toa fourth preferred embodiment of the present invention. The encoder andthe decoder establishes the video_object_layer_shape_extension which hasthree auxiliary components of the horizontal disparity type, thevertical disparity type, and the luminance residual texture type whichare additionally defined with reference to FIG. 4, and performs encodingand decoding.

That is, the pixel-based horizontal disparity map found with referenceto the left image is allocated to aux_comp_type[0], the pixel-basedvertical disparity map found with reference to the left image isallocated to aux_comp_type[1], and the luminance residual texture whichis residual image data on the luminance component is allocated toaux_comp_type[2]. The encoder and the decoder are usable for simpleimages which do not greatly influence deterioration of image qualitywhen performing encoding without chrominance residual textureinformation which is residual image data on the chrominance component,or for images which require no precise reconstruction.

As shown in FIG. 12, the encoder according to the fourth preferredembodiment of the present invention comprises a disparity estimator 100,a video object encoder 200, auxiliary component encoders 320, 340, and380, a disparity compensator 420, an adder 500, and a variable lengthencoder 600.

The components of FIG. 12 which perform the same or similar functions asthose of the encoder of FIG. 6 have the same reference numerals, and norepeated descriptions on the components of FIG. 12 which execute thesame operations as those of the components of FIG. 6 will be provided.

The disparity estimator 100 receives left and right images, and finds apixel-based horizontal disparity map and a vertical disparity map of theright image with reference to the left image.

The auxiliary component encoder 380 receives the vertical disparity mapoutput by the disparity estimator 100 and the motion vector output bythe video object encoder 200, generates a quantized vertical disparitymap, outputs the quantized vertical disparity map, and outputs avertical disparity map obtained by reconstructing the quantized verticaldisparity map.

The disparity compensator 420 performs disparity compensation based onpixel values (luminance) of the reconstructed left image output by thevideo object encoder 200, the horizontal disparity vectors of thereconstructed horizontal disparity map output by the auxiliary componentencoder 320, and the vertical disparity vectors of the reconstructedvertical disparity map output by the auxiliary component encoder 380,and outputs pixel values (luminance) of the compensated right image.

The adder 500 performs a residual process on the pixel value (luminance)of the right image and the pixel value (luminance) of thedisparity-compensated right image output by the disparity compensator420 to output luminance residual texture, and the auxiliary componentencoder 340 encodes the luminance residual texture to output quantizedluminance residual texture.

The variable length encoder 600 performs variable length encoding on thequantized video object and the motion vector output by the video objectencoder 200, the quantized horizontal disparity map output by theauxiliary component encoder 320, the quantized vertical disparity mapoutput by the auxiliary component encoder 380, and the quantizedluminance residual texture output by the auxiliary component encoder360, and outputs an encoding stream.

FIG. 14 shows an encoding stream output by the encoder according to thefourth preferred embodiment of the present invention. As shown, theencoding stream output by the encoder includes macro blocks of theencoded video object, macro blocks of the auxiliary component AC[0] forthe encoded horizontal disparity map, macro blocks of the auxiliarycomponent AC[1] for the encoded vertical disparity map, and macro blocksof the auxiliary component AC[2] for encoded luminance residual texture.

FIG. 13 shows a decoder according to the fourth preferred embodiment ofthe present invention.

As shown, the decoder comprises a variable length decoder 700, a videoobject decoder 800, auxiliary component decoders 920, 940, and 980, adisparity compensator 2000, and an adder 1100.

The components of FIG. 13 which perform the same or similar functions asthose of the decoder of FIG. 7 have the same reference numerals, and norepeated descriptions on the components of FIG. 13 which execute thesame operations as those of the components of FIG. 7 will be provided.

The variable length decoder 700 performs variable length decoding on theencoding stream output by the encoder as shown in FIG. 14, and outputsquantized data of the video object, a motion vector, quantized data ofthe horizontal disparity map, quantized data of the vertical disparitymap, and quantized data of the luminance residual texture.

The auxiliary component decoder 980 receives the quantized data of thevertical disparity map and the motion vector, and decodes the verticaldisparity map.

The disparity compensator 2000 receives a reconstructed left image, areconstructed horizontal disparity map, and a reconstructed verticaldisparity map, and performs disparity compensation based on disparityvectors of the horizontal disparity map and the vertical disparity map.

The adder 1100 adds the disparity-compensated luminance texture fromamong the data disparity-compensated by the disparity compensator 2000and the reconstructed luminance residual texture output by the auxiliarycomponent decoder 940 to reconstruct the luminance component of theright image.

FIGS. 15 and 16 respectively show an encoder and a decoder according toa fifth preferred embodiment of the present invention. The encoder andthe decoder establish the video_object_layer_shape_extension which hasfour auxiliary components of the horizontal disparity type, the verticaldisparity type, the luminance residual texture type, and the chrominanceresidual texture type which are additionally defined when the number ofauxiliary components of the MAC is increased to equal to or more thanfour as shown in FIG. 5, and performs encoding and decoding.

That is, the encoder and the decoder allocates the pixel-basedhorizontal disparity map found with reference to the left image toaux_comp_type[0], the pixel-based vertical disparity map found withreference to the left image to aux_comp_type[1], the luminance residualtexture which is residual image data on the luminance component toaux_comp_type[2], and the chrominance residual texture which is residualimage data on the chrominance component to aux_comp_type[3] to performencoding and decoding.

The above-described fifth preferred embodiment can, be applicable to thecase of reconstructing high-quality images by using all kinds ofauxiliary information on the right image.

As shown in FIG. 15, the encoder according to the fifth preferredembodiment of the present invention comprises a disparity estimator 100,a video object encoder 200, auxiliary component encoders 320, 340, 360,and 380, a disparity compensator 420, an adder 500, and a variablelength encoder 600.

The components of FIG. 15 which perform the same or similar functions asthose of the encoder of FIG. 12 have the same reference numerals, and norepeated descriptions on the components of FIG. 15 which execute thesame operations as those of the components of FIG. 12 will be provided.

The disparity estimator 100 receives left and right images, and finds apixel-based horizontal disparity map and a vertical disparity map of theright image with reference to the left image.

The auxiliary component encoder 380 receives the vertical disparity mapoutput by the disparity estimator 100 and the motion vector output bythe video object encoder 200, generates a quantized vertical disparitymap, outputs the quantized vertical disparity map, and outputs avertical disparity map obtained by reconstructing the quantized verticaldisparity map.

The disparity compensator 420 performs disparity compensation based onpixel values (luminance and chrominance) of the reconstructed left imageoutput by the video object encoder 200, the horizontal disparity vectorsof the reconstructed horizontal disparity map output by the auxiliarycomponent encoder 320, and the vertical disparity vectors of thereconstructed vertical disparity map output by the auxiliary componentencoder 380, and outputs pixel values (luminance and chrominance) of thecompensated right image.

The adder 500 performs a residual process on the pixel values (luminanceand chrominance) of the right image and the pixel values (luminance andchrominance) of the disparity-compensated right image output by thedisparity compensator 420 to output luminance residual texture andchrominance residual texture, and the auxiliary component encoder 340and the auxiliary component encoder 360 respectively encode theluminance residual texture and the chrominance residual texture tooutput quantized luminance residual texture and quantized chrominanceresidual texture.

The variable length encoder 600 performs variable length encoding on thequantized video object and the motion vector output by the video objectencoder 200, the quantized horizontal disparity map output by theauxiliary component encoder 320, the quantized vertical disparity mapoutput by the auxiliary component encoder 380, the quantized luminanceresidual texture output by the auxiliary component encoder 340, and thequantized chrominance residual texture output by the auxiliary componentencoder 360, and outputs an encoding stream.

FIG. 17 shows an encoding stream output by the encoder according to thefifth preferred embodiment of the present invention. As shown, theencoding stream output by the encoder includes macro blocks of theencoded video object, macro blocks of the auxiliary component AC[0] forthe encoded horizontal disparity map, macro blocks of the auxiliarycomponent AC[1] for the encoded vertical disparity map, macro blocks ofthe auxiliary component AC[2] for the encoded luminance residualtexture, and macro blocks of the auxiliary component AC[3] for theencoded chrominance residual texture.

FIG. 16 shows a decoder according to the fifth preferred embodiment ofthe present invention.

As shown, the decoder comprises a variable length decoder 700, a videoobject decoder 800, auxiliary component decoders 920, 940, 960, and 980,a disparity compensator 2000, and adders 1100 and 1200.

The components of FIG. 16 which perform the same or similar functions asthose of the decoder of FIG. 13 have the same reference numerals, and norepeated descriptions on the components of FIG. 16 which execute thesame operations as those of the components of FIG. 13 will be provided.

The variable length decoder 700 performs variable length decoding on theencoding stream output by the encoder and shown in FIG. 17, and outputsquantized data of the video object, a motion vector, quantized data ofthe horizontal disparity map, quantized data of the vertical disparitymap, quantized data of the luminance residual texture, and quantizeddata of the chrominance residual texture.

The disparity compensator 2000 receives a reconstructed left image, areconstructed horizontal disparity map, and a reconstructed verticaldisparity map, and performs disparity compensation based on disparityvectors of the horizontal disparity map and the vertical disparity map.

The adder 1100 adds the disparity-compensated luminance texture fromamong the data disparity-compensated by the disparity compensator 2000and the reconstructed luminance residual texture output by the auxiliarycomponent decoder 940 to reconstruct the luminance component of theright image.

The adder 1200 adds the disparity-compensated chrominance texture fromamong the data disparity-compensated by the disparity compensator 2000and the reconstructed chrominance residual texture output by theauxiliary component decoder 960 to reconstruct the chrominance componentof the right image.

The video object encoders 200, the auxiliary component encoders 320,340, 360, and 380, the video object decoder 800, and the auxiliarycomponent decoders 920, 940, 960, and 980 will now be described infurther detail.

FIG. 18 shows a detailed block diagram for the video object encoder 200according to the preferred embodiments of the present invention.

As shown, the video object encoder 200 comprises an encoding unit 220, adecoding unit 240, a motion estimator 260, and a motion compensator 280.

The encoding unit 220 performs a discrete cosine transform on the videoobject data (left image) and the motion-compensated data havingundergone the residual process, quantizes thediscrete-cosine-transformed data, and outputs the quantized videoobject. The encoding unit 220 comprises an adder 221 for performing aresidual process on the video object data and the motion-compensateddata, a DCT (discrete cosine transformer) 222 for performing a discretecosine transform on the residual data output by the adder 221, and aquantizer 223 for quantizing the data output by the DCT 222.

The decoding unit 240 dequantizes the quantized video object output bythe encoding unit 220, and performs an inverse discrete cosine transformon the de-quantized data to reconstruct the video object data. Thedecoding unit 240 comprises a dequantizer 241 for dequantizing thequantized video object output by the encoding unit 220, an IDCT (inversediscrete cosine transformer) 242 for performing an inverse discretecosine transform on the data output by the dequantizer 241, an adder 243for adding the video object output by the IDCT 242 and themotion-compensated data to reconstruct the video object data, and amemory 244 for storing the reconstructed left image output by the adder243.

The motion estimator 260 compares the video object data (left image)with the reconstructed left image of a previous frame stored in thememory 244 to output a motion vector MV.

The motion compensator 280 compares the motion vector output by themotion estimator 260 with the left image of a previous frame stored inthe memory 244 to output motion compensation data.

FIG. 19 shows a detailed block diagram for the auxiliary componentencoder 300 according to the preferred embodiments of the presentinvention.

As shown, the auxiliary component encoder 300 comprises an encoding unit310, a decoding unit 330, and a motion compensator 350.

The encoding unit 310 performs a discrete cosine transform on theauxiliary component data and the motion-compensated data havingundergone the residual process, quantizes thediscrete-cosine-transformed data, and outputs the quantized auxiliarycomponent data. The encoding unit 310 comprises an adder 311 forperforming a residual process on the auxiliary component data and themotion-compensated data, a DCT (discrete cosine transformer) 312 forperforming a discrete cosine transform on the residual data output bythe adder 311, and a quantizer 313 for quantizing the data output by theDCT 312.

The decoding unit 330 dequantizes the quantized auxiliary component dataoutput by the encoding unit 310, and performs an inverse discrete cosinetransform on the de-quantized data to reconstruct the auxiliarycomponent data. The decoding unit 330 comprises a dequantizer 331 fordequantizing the quantized auxiliary component data output by theencoding unit 310, an IDCT (inverse discrete cosine transformer) 332 forperforming an inverse discrete cosine transform on the data output bythe dequantizer 331, an adder 333 for adding the auxiliary componentdata output by the IDCT 332 and the motion-compensated data toreconstruct the auxiliary component data, and a memory 334 for storingthe reconstructed auxiliary component data output by the adder 333.

The motion compensator 350 compares the motion vector output by themotion estimator 260 of the video object encoder 200 with the auxiliarycomponent data of a previous frame stored in the memory 334 to outputmotion compensation data on the auxiliary component.

FIG. 20 shows a detailed block diagram for the video object decoder 800according to the preferred embodiments of the present invention.

As shown, the video object decoder 800 comprises a dequantizer 810, anIDCT 820, an adder 830, a motion compensator 850, and a memory 840.

The dequantizer 810 dequantizes the quantized data of the video objectoutput by the variable length decoder, the IDCT 820 performs an inversediscrete cosine transform on the data output by the dequantizer 810, andthe motion compensator 850 compares the reconstructed video object dataof a previous frame stored in the memory 840 with the motion vector tocompensate for motion, and outputs compensated motion vector data.

The adder 830 adds the video object output by the IDCT 820 and themotion-compensated vector data output by the motion compensator 850 tooutput reconstructed video object data.

FIG. 21 shows a detailed block diagram for the auxiliary componentdecoder 900 according to the preferred embodiments of the presentinvention.

As shown, the auxiliary component decoder 900 comprises a dequantizer901, an IDCT 902, an adder 903, a motion compensator 905, and a memory904.

The dequantizer 901 dequantizes the quantized data of the auxiliarycomponent output by the variable length decoder, the IDCT 902 performsan inverse discrete cosine transform on the data output by thedequantizer 901, and the motion compensator 905 compares thereconstructed auxiliary component data of a previous frame stored in thememory 904 with the motion vector to compensate for motion, and outputscompensated motion vector data.

The adder 903 adds the auxiliary component data output by the IDCT 902and the motion-compensated vector data output by the motion compensator905 to output reconstructed video object data.

As described, compatibility for using the current MPEG-4 encodingtechniques and systems is provided since the stereoscopic video areencoded by using the MAC of the MPEG-4 standard according to thepreferred embodiments of the present invention.

Also, synchronization of between the right and left images is simplyperformed since the encoding streams of the right and left images areoutput as a single encoding stream.

Further, encoding efficiencies are improved by allocating fourcomponents of auxiliary information of the right image to the MAC invarious ways, selecting thereof according to image quality levelsdesired by a user or a producer, and encoding the same.

While this invention has been described in connection with what ispresently considered to be the most practical and preferred embodiment,it is to be understood that the invention is not limited to thedisclosed embodiments, but, on the contrary, is intended to covervarious modifications and equivalent arrangements included within thespirit and scope of the appended claims.

For example, the auxiliary information is allocated using the MACdefined by the MPEG-4 standard, and in addition, information defined byother protocols can be used.

As described, compatibility for using the existing MPEG-4 encodingtechniques and systems is maintained, and complexity of synchronizationof between the right and left images is minimized.

In addition, encoding efficiency is improved by selectively controllingthe image quality and encoding it according to importance and complexityof the images.

1. A method for encoding stereoscopic video including first and secondimages, comprising: (a) encoding the first image, and outputting aquantized video object and a motion vector of the first image; (b)receiving the first and second images, and finding a pixel-basedhorizontal disparity map on the second image with reference to the firstimage; and (c) encoding the horizontal disparity map and outputting aquantized horizontal disparity map based on the pixel-based horizontaldisparity map and a motion vector; and (d) performing variable lengthencoding on the quantized video object, the motion vector, and thequantized horizontal disparity map, and outputting the quantized videoobject, the motion vector, and the quantized horizontal disparity map onwhich the variable length coding is performed as a single stream.
 2. Themethod of claim 1, wherein the quantized horizontal disparity map isallocated to an auxiliary component of a disparity type of the MPEG-4MAC (multiple auxiliary component) and is encoded.
 3. The method ofclaim 1, wherein the first image is a left image, and the second imageis a right image.
 4. A method for decoding stereoscopic video includingfirst and second images, comprising: (a) receiving an encoding stream,and outputting quantized data of a video object of the first image, amotion vector, and quantized data of a horizontal disparity map byperforming variable length decoding on said encoding stream: (b)decoding the video object and reconstructing the first image based onthe quantized data of the video object and the motion vector; (c)decoding the quantized data of the horizontal disparity map based on thequantized data of the horizontal disparity map and the motion vector;and (d) performing disparity compensation based on the reconstructedfirst image and the decoded horizontal disparity map, and reconstructingthe second image.
 5. The method of claim 4, wherein the first image is aleft image, and the second image is a right image.
 6. A method forencoding stereoscopic video including first and second images,comprising: (a) encoding the first image, and outputting a quantizedvideo object and a motion vector of the first image; (b) decoding thequantized video object output in (a), and reconstructing the firstimage; (c) receiving the first and second images, and finding apixel-based horizontal disparity map on the second image with referenceto the first image; (d) encoding the horizontal disparity map andoutputting a quantized horizontal disparity map based on the pixel-basedhorizontal disparity map and the motion vector; (e) reconstructing thequantized horizontal disparity map output in (d), and outputting areconstructed horizontal disparity map; (f) performing disparitycompensation and outputting a pixel value of a disparity-compensatedsecond image based on a pixel value of the first image reconstructed in(b) and a horizontal disparity vector value of the horizontal disparitymap reconstructed in (e); and (g) performing a residual process on thepixel value of the second image and the pixel value of thedisparity-compensated second image output in (f) to output luminanceresidual texture, and encoding the luminance residual to outputquantized luminance residual texture; and (h) performing variable lengthencoding on the quantized video object, the motion vector, the quantizedhorizontal disparity map, and the quantized luminance residual texture,and outputting the quantized video object, the motion vector, thequantized horizontal disparity map, and the quantized luminance residualtexture on which the variable length coding is performed as a singlestream.
 7. The method of claim 6, wherein the quantized horizontaldisparity map and the quantized luminance residual texture are allocatedto the MPEG-4 MAC and then encoded.
 8. The method of claim 6, whereinthe first image is a left image, and the second image is a right image.9. The method of claim 6, further comprising (h) performing a residualprocess on the pixel value of the second image and the pixel value ofthe disparity-compensated second image output in (f) to outputchrominance residual texture, and encoding the chrominance residualtexture to output quantized chrominance residual texture.
 10. The methodof claim 9, further comprising (i) performing variable length encodingon the quantized video object, the motion vector, the quantizedhorizontal disparity map, the quantized luminance residual texture, andthe quantized chrominance residual texture, and outputting the quantizedvideo object, the motion vector, the quantized horizontal disparity map,the quantized luminance residual texture, and the quantized chrominanceresidual texture on which the variable length encoding is performed as asingle stream.
 11. The method of claim 9, wherein the quantizedhorizontal disparity map, the quantized luminance residual texture, andthe quantized chrominance residual texture are allocated to the MPEG-4MAC and then encoded.
 12. A method for decoding stereoscopic videoincluding first and second images, comprising: (a) receiving an encodingstream, and outputting quantized data of a video object of the firstimage, a motion vector, quantized data of a horizontal disparity map,and quantized data of luminance residual texture by performing variablelength decoding on said encoding stream. (b) decoding the video objectand reconstructing the first image based on the quantized data of thevideo object and the motion vector; (c) decoding the quantized data ofthe horizontal disparity map based on the quantized data of thehorizontal disparity map and the motion vector; (d) decoding thequantized data of the luminance residual texture based on the quantizeddata of the luminance residual texture and the motion vector; (e)performing disparity compensation based on the reconstructed first imageand the decoded horizontal disparity map, and outputtingdisparity-compensated luminance texture; and (f) adding thedisparity-compensated luminance texture and the luminance residualtexture reconstructed in (d) to reconstruct the second image.
 13. Themethod of claim 12, wherein the first image is a left image, and thesecond image is a right image.
 14. The method of claim 12, wherein thestep (a) comprises receiving the encoding stream, and additionallyoutputting quantized data of chrominance residual texture, the step (d)comprises additionally decoding the quantized data of chrominanceresidual texture based on the quantized data of chrominance residualtexture and the motion vector, and the step (f) comprises additionallyadding the disparity-compensated chrominance texture and the chrominanceresidual texture to reconstruct the second image.
 15. A method forencoding stereoscopic video including first and second images,comprising: (a) encoding the first image, and outputting a quantizedvideo object and a motion vector of the first image; (b) decoding thequantized video object output in (a), and reconstructing the firstimage; (c) receiving the first and second images, and finding apixel-based horizontal disparity map and a pixel-based verticaldisparity map on the second image with reference to the first image; (d)encoding the horizontal disparity map and outputting a quantizedhorizontal disparity map based on the pixel-based horizontal disparitymap and the motion vector, (e) encoding the vertical disparity map andoutputting a quantized vertical disparity map based on the pixel-basedvertical disparity map and the motion vector; (f) reconstructing thequantized horizontal disparity map output in (d), and outputting areconstructed horizontal disparity map; (g) reconstructing the quantizedvertical disparity map output in (d), and outputting a reconstructedvertical disparity map; (h) performing disparity compensation andoutputting a pixel value of a disparity-compensated second image basedon a pixel value of the first image reconstructed in (b), a horizontaldisparity vector value of the horizontal disparity map reconstructed in(f), and a vertical disparity vector value of the vertical disparity mapreconstructed in (h); and (i) performing a residual process on the pixelvalue of the second image and the pixel value of thedisparity-compensated second image output in (h) to output luminanceresidual texture, and encoding the luminance residual texture to outputquantized luminance residual texture; and (i) performing variable lengthencoding on the quantized video object, the motion vector, the quantizedhorizontal disparity map, the quantized vertical disparity map, and thequantized luminance residual texture, and outputting the quantized videoobject, the motion vector, the quantized horizontal map, the quantizedvertical disparity map, and the quantized luminance residual texture onwhich the variable length encoding is performed as a single stream. 16.The method of claim 15, wherein the quantized horizontal disparity map,the quantized vertical disparity map, and the quantized luminanceresidual texture are allocated to the MPEG-4 MAC and then encoded. 17.The method of claim 15, wherein the first image is a left image, and thesecond image is a right image.
 18. The method of claim 15, furthercomprising (j) performing a residual process on the pixel value of thesecond image and the pixel value of the disparity-compensated secondimage output in (h) to output chrominance residual texture, and encodingthe chrominance residual texture to output quantized chrominanceresidual texture.
 19. The method of claim 18, further comprising (k)performing variable length encoding on the quantized video object, themotion vector, the quantized horizontal disparity map, the quantizedvertical disparity map, the quantized luminance residual texture, andthe quantized chrominance residual texture, and outputting outputtingthe quantized video object, the motion vector, the quantized horizontaldisparity map, the quantized vertical disparity map, the quantizedluminance residual texture, and the quantized chrominance residualtexture on which the variable length coding is performed as a singlestream.
 20. A method for decoding stereoscopic video including first andsecond images, comprising: (a) receiving an encoding stream, andoutputting quantized video data of a video object of the first image, amotion vector, quantized data of a horizontal disparity map, quantizeddata of a vertical disparity map, and quantized data of luminanceresidual texture by performing variable length decoding on the encodingstream. (b) decoding the video object and reconstructing the first imagebased on the quantized data of the video object and the motion vector;(c) decoding the quantized data of the horizontal disparity map based onthe quantized data of the horizontal disparity map and the motionvector, (d) decoding the quantized data of the vertical disparity mapbased on the quantized data of the vertical disparity map and the motionvector; (e) decoding the quantized data of the luminance residualtexture based on the quantized data of the luminance residual textureand the motion vector; (f) performing disparity compensation based onthe reconstructed first image, the decoded horizontal disparity map, andthe decoded vertical disparity map, and outputting disparity-compensatedluminance texture; and (g) adding the disparity-compensated luminancetexture and the luminance residual texture reconstructed in (e) toreconstruct the second image.
 21. The method of claim 20, wherein thestep (a) comprises receiving the encoding stream, and additionallyoutputting quantized data of chrominance residual texture; the step (e)comprises additionally decoding the quantized data, of chrominanceresidual texture based on the quantized data of chrominance residualtexture and the motion vector, and the step (g) comprises additionallyadding the disparity-compensated chrominance texture and the chrominanceresidual texture to reconstruct the second image.
 22. An encoder forstereoscopic video including first and second images, comprising: avideo object encoder for encoding the first image, and outputting aquantized video object and a motion vector of the first image; adisparity estimator for receiving the first and second images, andfinding a pixel-based horizontal disparity map on the second image withreference to the first image; and An auxiliary component encoder forencoding the horizontal disparity map and outputting a quantizedhorizontal disparity map based on the pixel-based horizontal disparitymap output by the disparity estimator and a motion vector output by thevideo object encoder; and a variable length encoder for performingvariable length encoding on the quantized video object, the motionvector, and the quantized horizontal disparity map, and outputting thequantized video object, the motion vector and the quantized horizontaldisparity map on which the variable length coding is performed as asingle stream.
 23. A decoder for stereoscopic video including first andsecond images, comprising: A variable length decoder for receiving anencoding stream, and performing variable length decoding on saidencoding stream to output quantized data of video object ob the firstimage, a motion vector, and quantized data of a horizontal disparitymap; a video object decoder for decoding the video object andreconstructing the first image based on the quantized data of the videoobject and the motion vector output by the variable length decoder, anauxiliary component decoder for decoding the quantized data of thehorizontal disparity map based on the quantized data of the horizontaldisparity map and the motion vector output by the variable lengthdecoder; and a disparity compensator for performing disparitycompensation based on the reconstructed first image output by the videoobject decoder and the decoded horizontal disparity map output by theauxiliary component decoder, and reconstructing the second image.
 24. Anencoder for stereoscopic video including first and second images,comprising: a video object encoder for encoding the first image tooutput a quantized video object and a motion vector of the first image,and encoding the quantized video object to output a reconstructed firstimage; a disparity estimator for receiving the first and second images,and finding a pixel-based horizontal disparity map on the second imagewith reference to the first image; a first auxiliary component encoderfor encoding the horizontal disparity map to output a quantizedhorizontal disparity map, and decoding the output and quantizedhorizontal disparity map to output a reconstructed horizontal disparitymap based on the pixel-based horizontal disparity map output by thedisparity estimator and the motion vector output by the video objectencoder, a disparity compensator for performing disparity compensationand outputting a pixel value of a disparity-compensated second imagebased on a pixel value of the reconstructed first image output by thevideo object encoder and a horizontal disparity vector value of the,reconstructed horizontal disparity map output by the first auxiliarycomponent encoder, and a second auxiliary component encoder forperforming a residual process on the pixel value of the second image andthe pixel value of the disparity-compensated second image output buy thedisparity compensator to output luminance residual texture, and encodingthe luminance residual texture to output quantized luminance residualtexture; and a variable length encoder for performing variable lengthencoding on the quantized video object, the motion vector, the quantizedhorizontal disparity map, and the quantized luminance residual texture,and outputting the quantize video object, the motion vector, thequantized horizontal disparity map, and the quantized luminance residualtexture on which the variable length coding is performed as a singlestream.
 25. The encoder of claim 24, wherein the quantized horizontaldisparity map and the quantized luminance residual texture are allocatedto the MPEG-4 MAC and then encoded.
 26. The encoder of claim 24, furthercomprising a third auxiliary component encoder for performing a residualprocess on the pixel value of the second image and the pixel value ofthe disparity-compensated second image output by the disparitycompensator to output chrominance residual texture, and encoding thechrominance residual texture to output quantized chrominance residualtexture.
 27. The encoder of claim 26, wherein the variable lengthencoder performing the variable length encoding on the quantizedchrominance residual texture in addition to the quantized video object,the motion vector, the quantized horizontal disparity map, and thequantized luminance residual texture, and outputting the quantized videoobject, the motion vector the quantized horizontal disparity map, thequantized luminance residual texture, and the quantized chrominanceresidual texture on which the variable length coding is performed as thesingle stream.
 28. The encoder of claim 26, wherein the quantizedhorizontal disparity map, the quantized luminance residual texture, andthe quantized chrominance residual texture are allocated to the MPEG-4MAC and then encoded.
 29. The encoder of claim 24, wherein the disparityestimator additionally outputs a pixel-based vertical disparity map onthe second image with reference to the first image; the encoder forstereoscopic video further comprises a third auxiliary component encoderfor encoding the vertical disparity map and outputting a quantizedvertical disparity map based on the pixel-based vertical disparity mapoutput by the disparity estimator and the motion vector output by thevideo object encoder; and the disparity compensator performs disparitycompensation and outputs a pixel value of the disparity-compensatedsecond image based on the pixel value of the reconstructed first image,a horizontal disparity vector value of the reconstructed horizontaldisparity map, and the reconstructed vertical disparity map.
 30. Theencoder of claim 29, wherein the variable length encoder performs thevariable length encoding on the quantized vertical disparity map inaddition to the quantized video object, the motion vector, the quantizedhorizontal disparity map, and the quantized luminance residual texture,and outputting the quantized video object, the motion vector, thequantized horizontal disparity map, the quantized vertical disparitymap, and the quantized luminance residual texture on which the variablelength coding is performed as the single stream.
 31. The encoder ofclaim 29, wherein the quantized horizontal disparity map, the quantizedvertical disparity map, and the quantized luminance residual texture areallocated to the MPEG-4 MAC and then encoded.
 32. The encoder of claim29, further comprising a fourth auxiliary component encoder forperforming a residual process on the pixel value of the second image andthe pixel value of the disparity-compensated second image output by thedisparity compensator to output chrominance residual texture, andencoding the chrominance residual texture to output quantizedchrominance residual texture.
 33. The encoder of claim 32, wherein thevariable length encoder performs the variable length encoding on thequantized chrominance residual texture in addition to the quantizedvideo object, the motion vector, the quantized horizontal disparity map,the quantized vertical disparity map, and the quantized luminanceresidual texture, and outputting the quantized video object, the motionvector, the quantized horizontal disparity map, the quantized verticaldisparity map, the quantized luminance residual texture, and thequantized chrominance residual texture on which the variable lengthcoding is performed as the single stream.
 34. The encoder of claim 24,wherein the video object encoder comprises: an encoding unit forperforming a residual process on the first image and the motioncompensated data, performing discrete cosine transform and quantizationon the data, and outputting a quantized video object; a decoding unitfor performing dequantization and inverse discrete cosine transform onthe quantized video object output by the encoding unit, reconstructingthe video object data, and storing the reconstructed video object datain a memory; a motion estimator for comparing the first image with thereconstructed video object data of a previous frame stored in thememory, and outputting a motion vector; and a motion compensator forcomparing the motion vector output by the motion estimator with thereconstructed video object data of a previous frame stored in thememory, and outputting motion compensation data.
 35. The encoder ofclaim 34, wherein the first auxiliary component encoder comprises: anencoding unit for performing a residual process on the horizontaldisparity map and the motion compensated data, performing discretecosine transform and quantization on the data, and outputting aquantized horizontal disparity map; a decoding unit for performingdequantization and inverse discrete cosine transform on the quantizedhorizontal disparity map output by the encoding unit, reconstructing thehorizontal disparity map, and storing the reconstructed horizontaldisparity map in a memory; and a motion compensator for comparing themotion vector output by the motion estimator of the video object encoderwith the reconstructed horizontal disparity map of a previous framestored in the memory, and outputting motion compensation data.
 36. Adecoder for stereoscopic video including first and second images,comprising: a variable length decoder for receiving an encoding stream,and perform variable length decoding on said encoding stream to outputquantized data of a video object of the first image, a motion vector,quantized data of a horizontal disparity map, and quantized data ofluminance residual texture; a video object decoder for decoding thevideo object and reconstructing the first image based on the quantizeddata of the video object and the motion vector; a first auxiliarycomponent decoder for decoding the quantized data of the horizontaldisparity map based on the quantized data of the horizontal disparitymap and the motion vector; a second auxiliary component decoder fordecoding the quantized data of the luminance residual texture based onthe quantized data of the luminance residual texture and the motionvector a disparity compensator for performing disparity compensationbased on the reconstructed first image output by the video objectdecoder and the decoded horizontal disparity map output by the firstauxiliary component decoder, and outputting disparity-compensatedluminance texture and disparity-compensated chrominance texture; and afirst adder for adding the disparity-compensated luminance textureoutput by the disparity compensator and the reconstructed luminanceresidual texture output by the second auxiliary component decoder. 37.The decoder of claim 36, wherein the variable length decoderadditionally outputs quantized data of chrominance residual-texture, andthe decoder for stereoscopic video further comprises: a third auxiliarycomponent decoder for decoding the quantized data of chrominanceresidual texture based on the quantized data of the chrominance residualtexture and the motion vector output, by the variable length decoder,and a second adder for adding the disparity-compensated chrominancetexture output by the disparity compensator and the reconstructedchrominance residual texture output by the third auxiliary componentdecoder.
 38. The decoder of claim 36, wherein the variable lengthdecoder additionally outputs quantized data of a vertical disparity map;the decoder for stereoscopic video further comprises a third auxiliarycomponent decoder for decoding the quantized data of the verticaldisparity map based on the quantized data of the vertical disparity mapand the motion vector output by the variable length decoder; and thedisparity compensator performs disparity compensation based on thereconstructed first image output by the video object decoder, thedecoded horizontal disparity map output by the first auxiliary componentdecoder, and the decoded vertical disparity map output by the thirdauxiliary component decoder, and outputs the disparity-compensatedluminance texture and the disparity-compensated chrominance texture. 39.The decoder of claim 38, wherein the variable length decoderadditionally outputs quantized data of chrominance residual texture, andthe decoder for stereoscopic video further comprises: a fourth auxiliarycomponent decoder for decoding the quantized data of chrominanceresidual texture based on the quantized data of the chrominance residualtexture and the motion vector output by the variable length decoder, anda second adder for adding the disparity-compensated chrominance textureoutput by the disparity compensator and the reconstructed chrominanceresidual texture output by the third auxiliary component decoder. 40.The decoder of claim 36, wherein the video object decoder comprises: adequantizer for dequantizing the quantized data of the video objectoutput by the variable length decoder; an IDCT (inverse discrete cosinetransformer) for performing inverse discrete cosine transform on thedata output by the dequantizer; a motion compensator for comparing thereconstructed video object data of a previous frame with themotion'vector to compensate for motion, and outputting a motion vector;and an adder for adding the video object output by the IDCT and themotion compensated data output by the motion compensator.
 41. Thedecoder of claim 36, wherein the first auxiliary component decodercomprises: a dequantizer for dequantizing the quantized data of thehorizontal disparity map output by the variable length decoder; an IDCT(inverse discrete cosine transformer) for performing inverse discretecosine transform on the data output by the dequantizer, a motioncompensator for comparing the reconstructed horizontal disparity map ofa previous frame with the motion vector to compensate for motion, andoutputting a motion vector; and an adder for adding the horizontaldisparity map output by the IDCT and the motion compensated data outputby the motion compensator.